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(54) Title: AN ACTIVE ELECTRO-OPTICAL DEVICE FOR DETECTING OBSTACLES, IN PARTICULAR FOR 
AUTONOMOUS NAVIGATION 

(57) Abstract: An electro-optical system capable of being embarked aboard 
mobile ground or flying units, to determine the optical flow generated by ob- 
stacles in relative motion with respect to the mobile unit. The system com- 
prises radiation (5), receiver means (1) for converting the radiation reflected 
by the objects into electrical signals and means (8) for processing the signals 
generated by the receiver means. The receiver means (1) are based on vi- 
sion sensors with matrix configuration. The emitter means (5, 6) shape the 
radiation beam in such a way that the radiation reflected by the objects and 
collected by the receiver means impacts at least on a part of the receiver ma- 
trix. The processing means compute the optical flow only on the elements of 
the receiver matrix that are impacted by the radiation. 
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"An active electro-optical device for detecting 
obstacles, . in particular for autonomous navigation 
systems" • 

* * * 

5 The present invention relates to an active electro- 

optical device for detecting obstacles, in particular 
for use in autonomous navigation systems for ground 
moving or flying units with multiple degrees of 
freedom. 

10 In particular, the invention relates to a device 

for detecting obstacles based on the measurement of the 
optical flow. 

The effectiveness of an autonomous navigation 
system depends on its ability to reach a determined 
IS position and/or orientation in space. One of the main 
problems for a device of this kind consists of 
perceiving the surrounding environment and reacting in 
timely fashion. 

Electro-optical devices for detecting obstacles 
SO based on the measurement of the optical flow have 
already been proposed. The optical flow indicates the 
speed whereby elements with different contrast move in 
a scene, as a result of the relative motion between an 
observer and the objects present in the scene. If one 
25 considers an object moving with a relative speed v with 
respect to an observer and at an orthogonal distance d 
with respect to the observer, the optical flow is given 
by the ratio between the speed v and the distance d. 

Therefore, the optical flow measurement depends on 
30 the following factors: 

resolution, field of view, frame-rate and 
sensitivity of the receiving means, 

distance between the receiver means and the 
objects present in the scene, 
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- relative speed between the receiver means and the 
objects present in the scene. 

Electro-optical devices for detecting obstacles 
based on the measurement of the optical flow are 
£ described for example in the following patent 
documents: US 5717792, US 5257209, EP 631109, US 
5798796 and EP 436213" 

Algorithms for measuring the optical flow are 
typically implemented in passive electro-optical 

ID devices, i.e. devices which detect the radiation 
emitted by sources not included in the system (for 
instance, sun, moon, artificial lighting system, etc.) 
and reflected by the scene. To simplify computations, 
typically a small number of distinct sensors are used, 

IS whereon the optical flow is computed. The use of 
receiver means constituted, by a great number of 
sensitive elements (for instance CCD or CMOS vision 
sensors) would require powerful computational units to 
implement the artificial vision algorithms. It must be 

ED kept in mind that the scene viewed by a passive vision 
sensor is usually very complex, since it includes 
objects positioned at a distance which may vary from a 
few centimetres to infinity. The scene therefore 
generates an enormous quantity of information about the 

E5 optical flow of the objects present. The video 
information of the scene acquired by the optical sensor 
must then be filtered in the space and frequency domain 
to reduce the quantity and complexity of . the data. Only 
after the information has been conveniently processed 

3D can it be used for optical flow computation. 

. The object of the present invention is to provide 
an active electro-optical device for detecting 
obstacles which allows to reduce considerably the 
quantity of data on which optical flow algorithms for 

35 obstacle detection are to be implemented. 
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According to the present invention, said object is 
achieved by a device having the characteristics set out 
in the main claim. 

The present invention shall now be described in 
S detail with reference to the accompanying drawings, 
provided purely by way of non limiting example, in 
which: 

Figure 1 is a schematic view showing the 
operating principle of the device according to the 
ID present invention, 

- Figure 2 is a block diagram of the device of 
Figure 1 , 

- Figures 3a and 3b schematically show a scene and 
the manner whereby the scene is detected by the device 

IS according to the present invention, 

- Figures 4a, 4b, 5a, 5b, 6a, 6b, 7a and 7b are 
schematic views of a scene and the manner whereby said 
scene is detected by some embodiments of the device 
according to the present invention, 

ED - Figure 8 is a schematic view showing the 

arrangement of three devices according to the present 
invention for measuring the frontal and lateral optical 
flow with respect to the direction of motion, and 

- Figure 9 is a block diagram of an autonomous 
E5 navigation system using an electro-optical device 

according to the present invention. 

Figures 1 and 2 show the operating principle of the 
device according to the present invention. Said device 
comprises radiation emitter means 5, radiation receiver 

3D means 1 which collect the radiation reflected by 
objects situated in the field of view (FOV) and means 8 
for processing the electrical signals generated by the 
receiver means 1. The radiation receiver means 1 can 
be, for example, CMOS or CCD optical sensors with pixel 

35 matrix configuration. The radiation emitter means 5 
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can, for example, be a semiconductor laser or a LED 
with narrow-band emission, centred around a wavelength 
in the visible, infrared or ultraviolet range. The 
radiation generated by the emitter means 5 is 
5 designated by the reference number 7 and the radiation 
reflected by the objects present in the field of view 
is designated by the number 4. According to a 
characteristic of the present invention, the radiation 
beam 7 generated by the emitter means 5 is shaped in 

□ such a way that the reflected radiation 4 collected by 
the receiver means 1 impacts only on a predefined part 
of the receiver matrix. 

For example, the device according to the present 
invention can use a cylindrical lens 6 which generates 
5 an optical beam 7 of rectangular shape, so conformed 
that the radiation reflected by the objects and focused 
on the photosensor matrix by means of an image forming 
lens 3 impacts a single row (and/or column) of adjacent 
pixels or a set of adjacent row (and/or columns) . A 

□ diffraction grating (not shown) can be used in 
combination with the cylindrical lens 6 to generate a 
set of rectangular beams, mutually parallel and 
angularly separated, or to form the beam in complex 
geometric figures . 

5 To make the electro-optical device immune to the 

broad band radiation emitted or reflected by the 
background, the device can be provided with a band- pass 
optical filter 2 with narrow band, with the 
transmittance peak centred at the emission peak of the 

□ emitter means 5 . 

The maximum distance of the objects that contribute 
to the optical flow detected by the receiver means 1 is 
determined by the intensity of the radiation emitted by 
the emitter means 5 (and hence by the emission power of 
S the generated optical beam) , by the reflectance of the 
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objects impacted by the radiation and by the 
sensitivity of the receiver means 1. 

Optical flow algorithms are implemented on the 
processing means 8 to determine at what speed the 
5 obstacles evolve in the scene with respect to the 
mobile unit . 

The two-dimensional optical flow (F.O.) in a point 

(x,y) of the focal plane of the image acquired by the 

receiver means is the vector sum of an optical flow in 

ID the direction x and in the direction y, i.e. 

F.O. = (F.O.) x x + (F.O.) y y 

where x and y are the versors respectively in the 

directions x and y. 

The optical flow can be measured using various 
IS mathematical approaches (gradient methods, correlation 
based methods, space-time methods, etc.). In 
particular, the gradient method is based on estimation 
of the spatial derivatives I x (x,y), I y (x,y) and time 
derivatives I t (x,y) associated to each point of the 
ED acquired image and on the computation of the optical 
flow components on the basis of • the luminosity 
conservation equation: 

I x {O.F) x +I y (OJ?.) y +/,=0 

In the case of a one -dimensional optical flow, for 
ES instance if (O.F.)y = 0, the equation has only the 
unknown (O.F.)x computed as: 

X 

In the case of a two-dimensional optical flow, the 
equation with two unknowns is solved with iterative 
30 techniques which allow to compute the two components 
(O.F. ) y and (O.F. ) x . 

Figures 3a and 3b show an example of , operation of 
the electro-optical device according to the present 
invention. Figure 3a schematically shows a scene with 
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background at infinite distance in which two nearby 
objects are present, one with rectangular shape and the 
other one with circular shape. The device according to 
the present invention is made in such a way as to 
5 illuminate a single row of pixels in the field of view 
of the receiver means. The strip in Figure 3a 
corresponds to the field of view illuminated by the 
emitter means . 

Figure 3b shows the information acquired by the 

10 receiver means. At the field of view illuminated by the 
emitter means, the receiver means detect, over time, 
the presence of nearby obstacles. 

The processing means 8 compute the optical flow 
only on the basis of the signals provided by the 

IS elements of the matrix of receivers (pixels) impacted 
by the radiation. In the example shown in Figure 3, the 
radiation reflected by the objects and focused on the 
sensor matrix can impact a single row of sensors 
(pixels) or a sheaf of rows adjacent to each othe?r. 

20 The device according to the present invention 

allows to define the maximum distance of the objects 
which contribute to the computation of the optical 
flow. Said distance is determined by the intensity of 
the radiation emitted by the emitter means, by the 

BS reflectance of the objects impacted by the radiation 
and by the sensitivity of the receiver means. The 
device according to the present invention can therefore 
be calibrated in such a way as to limit to a 
predetermined value the maximum distance of the objects 

30 which contribute to the measured optical flow. In this 
way, the information about the other objects of the 
scene (including the background) is filtered optically. 
This allows to reduce enormously the quantity of data 
. on which the algorithms for determining the optical 

3S flow for obstacle detection are to be implemented. In 
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other words, the system acquires a simplified 
information of the scene in which only the obstacles 
positioned in determined portions of the field of view 
(defined by the shape of the emitted beam) and within a 
5 distance range defined by the power of the emitted 
beam, by the reflectance of the objects and by the 
sensitivity of the receiver means are highlighted. 

In the example shown in Figures 4 a and 4b, the 
radiation beam generated by the emitter is shaped in 

ID such a way that the radiation reflected by the objects 
and focused on the sensor matrix impacts a plurality of 
rows separated from each other. Each row can impact a 
single array of sensors (pixels) of the matrix or a 
sheaf of mutually adjacent sensor arrays. 

IS In the examples shown in Figures 3b and 4b, the 

radiation reflected by the objects and focused on the 
sensor matrix impacts a single line (Figure 3b) or a 
set of mutually parallel lines (Figure 4b) , parallel to 
the direction of motion indicated by the arrow 10 in 

20 Figures 3b and 4b. 

The radiation beam generated by the emitter can be 
shaped in such a way that the radiation reflected by 
the objects and focused on the sensor matrix impacts a 
single column of the sensor matrix or a sheaf of 

BS mutually adjacent columns. In the example shown in 
Figures 5a and 5b, the radiation beam generated by the 
emitter is shaped in such a way that the radiation 
reflected by the objects and focused on the sensor 
matrix impacts a plurality of mutually separated 

30 columns. Each column can impact a single array of 
sensors or a sheaf of mutually adjacent sensor arrays. 
In the example shown in Figure 5b, too, the radiation 
focused on the sensor matrix impacts one or more lines, 
parallel to the main direction of motion indicated by 

3 S the arrow 1 0 . 



WO 2004/081598 



8 



PCT/IB2004/000601 



In the example shown in Figures 6a and 6b, the 
radiation focused on the sensor matrix impacts a 
plurality of rows and a plurality of columns according 
to a general grid configuration. Each row and each 
S column can illuminate a single pixel array or a sheaf 
of mutually adjacent pixel arrays of the sensor matrix. 
In Figure 6b, the rows are parallel to a first 
component of the direction of motion 10 'and the columns 
are parallel to a second component of the direction of 

ID motion 10 9 9 . 

• In the additional example shown in Figures 7a and 
7b, the radiation reflected by the objects and focused 
on the sensor matrix impacts a single plurality of 
inclined lines, parallel to the direction of motion 10. 

IS As a further variant, the radiation focused on the 
sensor matrix could impact a set of slanted lines, each 
parallel to one of the main directions of motion. 

The radiation generated by the emitter can also be 
shaped in such a way that the radiation reflected by 

ED the objects and focused on the sensor matrix impacts a 
set of sheaves of lines, where the lines of each sheaf 
are mutually parallel and parallel to one of the 
components of the motion, whilst the sheaves are not 
parallel to each other. The different sheaves can be 

HS generated by as many distinct sources of radiation. The 
radiation sources for the different sheaves can have 
mutually different peak wavelengths. Moreover, the 
different sheaves of lines with the different 
wavelengths can be collected by distinct receivers. 

3D One ore more active electro-optical devices 

according to the present invention can be used as a 
device for detecting obstacles in autonomous navigation 
systems . Figure 8 shows an example of a system with 
three electro-optical devices 9 for measuring the 

35 optical flow of the front area of the two lateral views 
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of a mobile unit with the direction of motion indicated 
by the arrow 10. In the example of Figure 8, the fields 
of view of the individual electro-optical devices 9 do 
not overlap. Alternatively, the devices 9 can be 
5 positioned in such a way that the respective fields of 
view mutually overlap at least partially. 

One or more electro-optical devices according to 
the invention can be integrated with inertial measuring 
and locating devices to obtain integrated systems for 

ID the autonomous stabilisation and navigation of ground 
moving or flying platforms. Figure 9 schematically 
shows an autonomous navigation system comprising a 
microprocessor computing unit 14, inertial measuring 
devices 11 , course indicating devices 13, satellite 

15 positioning devices 12 and one or- more electro-optical 
devices 9 of the type described above. The inertial 
measuring system 11 can be of the "strap-down" type 
with six degrees of freedom and constituted by three 
gyroscopes and three accelerometers with the axes 

ED oriented in three orthogonal directions. The course 
indicating unit 13 can employ a three—axis magnetometer 
which measures the Earth's magnetic field. The 
satellite positioning unit 12 can be constituted by a 
GPS (or Galileo) receiver and antenna. 

25 Naturally, without altering the principle of the 

invention, the construction details and the embodiments 
may be widely varied relative to what is described and 
illustrated purely by way of example herein, without 
thereby departing from the scope of the present 

3D invention as defined in the claims that follow. 
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CLAIMS 

1. An electro-optical system capable of being 
embarked aboard ground-moving or flying units, for 
determining the optical flow generated by obstacles in 

5 relative motion with respect to the mobile unit, 
characterised in that it comprises radiation emitter 
means (5), receiver means (1) for converting the 
radiation reflected by objects into electrical signals 
and means. (8) for processing the signals generated by 

ID said receiver means (1) , in which said receiver means 
(1) comprise at least a vision sensor with receiver 
matrix configuration and in which said emitter means 
(5, 6) conform the radiation beam in such a way that 
the radiation reflected by objects and collected by the 

IS receiver means impacts at least a part of the receiver 
matrix, and in which said processing means compute the 
optical flow only on the elements of the receiver 
matrix that are impacted by the radiation.. 

2. A system as claimed in claim 1, characterised in 
SO that the optical flow is determined within a 

predetermined distance range . 

3 . A system as claimed in claim 2 , characterised in 
that the maximum distance of the objects that 
contribute to the optical flow is determined by the 

BS intensity of the radiation emitted by the emitter means 
(5) , by the reflectance of the objects impacted by the 
radiation and by the sensitivity of the receiver means 
(1) . 

4. A system as claimed in claim 1, characterised in 
3D that the distribution and the shape of the receivers 

within the matrix are linked to the shape of the beam 
of radiation emitted by the emitter means (5) . 

5. A system as claimed in claim 1, characterised in 
that it comprises a band-pass optical filter with 
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narrow band (2) , with the transmittance peak centred at 
the emission peak of the emitter means (5) . 

6. A system as claimed in claim 1, characterised in 
that said sensor means (1) comprise a matrix of CCD or 

5 CMOS sensors . 

7. A system as claimed in claim 1, characterised in 
that the radiation beam (7) generated by the emitter 
means (5) is shaped in such a way that the radiation 
(4) reflected by the objects and focused on the sensor 

ID matrix (1) impacts a single array of sensors or a sheaf 
of adjacent sensor arrays. 

8. A system as claimed in claim 1, characterised in 
that the radiation beam (7) generated by the emitter 
means (5, 6)) is shaped in such a way that the 

IS radiation (4) reflected by the objects and focused on 
the sensor matrix (1) impacts a set of mutually 
separate rows . 

9. A system as claimed in claim 1, characterised in 
that the radiation beam (7) generated by said emitter 

2D means (5, 6)) is shaped in such a way that the 
radiation (4) reflected by the objects and focused on 
the sensor matrix (1) impacts a set of sheaves of rows, 
where the rows of each sheaf are mutually adjacent and 
the sheaves are separate from each other. 

SS 10. A system as claimed in claim 1, characterised 

in that the radiation beam (7) generated by said 
emitter means (5, 6) is shaped in such a way that the 
radiation (4) reflected by the objects and focused on 
the sensor matrix (1) impacts a single column or a 

3D sheaf of adjacent columns. 

11. A system as claimed in claim 1, . characterised 
in that the radiation beam (7) generated by said 
emitter means (5, 6) is shaped in such a way that the 
radiation (4) reflected by the objects and focused on 
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the sensor matrix (1) impacts a plurality of mutually 
separate columns . 

12. A system as claimed in claim 1, characterised 
in that the radiation beam (7) generated by said 
5 emitter means (5, 6)) is shaped in such a way that the 
radiation (4) reflected by the objects and focused on 
the sensor matrix (1) impacts a set of sheaves of 
columns, where the columns of each sheaf are mutually 
adjacent and the sheaves are separate from each other. 

ID 13 . A system as claimed in claim 1, characterised 

in that the radiation beam (7) generated by said 
emitter means (5, 6) is shaped in such a way that the 
radiation (4) reflected by the objects and focused on 
the sensor matrix (1) impacts a single line or a set of 

IS mutually parallel lines, parallel to the main direction 
of motion (10) . 

14. A system as claimed in claim 1, characterised 
in that the radiation beam (7) generated by said 
emitter means (5 #<cr 6)) is shaped in such a way that the 

SP radiation (4) reflected by the objects and focused on 
the sensor matrix (1) impacts a set of canted lines, 
each parallel to one of the main directions of motion. 

15. A system as claimed in claim 1, characterised 
in that the radiation beam (7) generated by said 

25 emitter means (5, 6) is shaped in such a way that the 
radiation (4) reflected by the objects and focused on 
the sensor matrix (1) impacts a set of sheaves of 
lines, where the lines of each sheaf are parallel to 
each other and parallel to one of the components of the 

3D motion, whilst the beams are not parallel to each 
other . 

16. A system as claimed in claim 15, characterised 
in that the different sheaves of lines are generated by 
as many distinct radiation sources. 
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17. A system as claimed in claim 16, characterised 
in that the radiation sources for the different sheaves 
have a different peak wavelength. 

18. A system as claimed in claim 17, characterised 
5 in that the different sheaves of lines with the 

different wavelengths are collected by distinct 
receivers. 

19. An autonomous navigation device comprising a 
plurality of systems according to one or more of the 

ID previous claims, each oriented in a different spatial 
direction, angularly separated from the others, so that 
the fields of view of the individual electro-optical 
systems do not overlap. 

20. A device for detecting obstacles comprising a 
IS plurality of systems according to one or more of the 

claims 1 through 18, each oriented in a different 
spatial direction, angularly separated from the others, 
so that the fields of view of the individual electro- 
optical systems mutually overlap at least partially. 

ED 21. An anti -collision system or an autonomous 

navigation system for mobile units, characterised in 
that it comprises and electro-optical system for 
measuring the optical flow according to one or more of 
the claims 1 through 18. 

25 22 . A system as claimed in claim 21, characterised 

in that it comprises a u strap-down" inertial navigation 
device . 

23. A system as claimed in claim 22, characterised 
in that the inertial navigation device comprises three 

30 gyroscopes, three accelerometers and/or a magnetometer 
with three axes used as course indicator and/or a 
satellite positioning system. 

24. An electro-optical according to claim 1, 
wherein the algorithms for determining the optical flow 
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are implemented on a processor using VLSI (Very Large 
Scale Integration) technology. 

25. An electro-optical system as claimed in claim 
1, characterised in that it comprises an optical device 
5 for measuring the distances of obstacles. 
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